release 0.6.6 — swarm health under CGNAT - #16

…check

BEP 3 verifies a piece against the SHA-1 in the metainfo, but a failure only
told us the piece was bad, never who sent it: the worker dropped the data,
re-requested every block, and a peer serving garbage was picked again. One
torrent sat at 99.84% for hours re-downloading the same index.

A piece now remembers which peer supplied each block, so a failure whose
blocks all came from one peer is reported to that peer's controller. A peer
stops being asked for an index it has already corrupted, and is dropped once
it has ruined @max_hash_failures distinct pieces — distinct, because retrying
one poisoned piece must not count as several offences.

Co-authored-by: Cursor <cursoragent@cursor.com>

Whether a piece can still be sourced is a question about that index, not
about the torrent. The abort check also required the swarm to be empty, so
on any torrent with peers a worker whose holders had all disconnected was
never released: it held one of the @max_parallel_pieces slots with no peer
monitored and its full block list unclaimed, forever.

Live this was 7 of 12 slots, capping a torrent with 26 unchoked peers at
5 pieces in flight. Ask whether any connected peer still has this index.

Co-authored-by: Cursor <cursoragent@cursor.com>

"Does this piece still have blocks" and "does it have blocks for you" are
different questions, and piece_has_waiting?/2 answered the first: it counts
blocks in flight to other peers. Outside endgame such a block cannot be
handed to a second peer, so a peer pinned to a fully claimed piece just sat
there until something else moved it.

Live: 27 of 37 peers pinned to 4 claimed pieces while 8 active pieces with
49-64 free blocks had no peer at all. Add piece_has_unclaimed?/2 and use it
both to refuse a repin onto such a piece and to free a pin that has drained.
Endgame keeps the old test, since duplicating in-flight blocks is its point.

Co-authored-by: Cursor <cursoragent@cursor.com>

A {:next_piece} that cannot place a piece reschedules itself, and every
independent trigger started its own such chain: the 2s reconcile tick, each
peer handoff kick, every piece's requests_are_dealt closure. Nothing
deduplicated them, so the chains accumulated. A live debug build went from
99 to about 2600 discovery dial cycles per minute over twelve minutes with
no change in swarm size, and the tracker began answering 403.

Every wake runs the same full pump attempt, so one pending wake per
controller is enough. Route them all through one timer, cancelling the
previous, and fold the queued strategy into the new one.

Co-authored-by: Cursor <cursoragent@cursor.com>

A TCP socket defaults to send_timeout: :infinity, so once a peer stopped
draining its receive window our sender blocked inside :prim_inet.send/4 and
never came back. Live one Peer.Sender held a 20863-message mailbox that only
grows: every block, have and keepalive queues behind a write that will not
complete.

Give both accepted and dialled sockets a 30s send timeout and close on it,
so the write fails and Peer.Sender.do_send/2 can stop the connection. 30s is
long enough for a genuinely slow peer on a congested path, short enough that
a dead one cannot park memory indefinitely.

Co-authored-by: Cursor <cursoragent@cursor.com>

…e task

The delivery task only tolerated :noproc, so a peer that shut down mid-call
or was too slow to accept the block took the task down with it: 333 [error]
crash reports in fifteen minutes, all restating one fact once per in-flight
block, drowning the log we read to find real faults.

Treat "this block is never reaching that peer" as a cancellation and log one
debug line. A timeout counts, because it means the peer's socket has stopped
draining; BEP 3 lets us simply not answer a request, and the peer will ask
again if it still wants it. Any other exit still crashes, so a genuine fault
in the delivery path is not swallowed.

Co-authored-by: Cursor <cursoragent@cursor.com>

…fill

Three compounding effects starved the swarm. Live, eight of nine torrents sat
at 1-3 connected peers while discovery knew 37-53 candidates each: only 3-8
were dialable, and the scarce v6 pool — the only family with real yield on this
host — was blocked 2-6 of 5-8. With 1-3 peers and no pieces to trade nothing
unchokes us, so nothing downloaded at all.

Only mark_productive/3 cleared a row, which asks "was this endpoint useful"
rather than "is it reachable". A leecher with nothing to trade is kept choked,
so a reachable peer never delivers bytes and carries its failure history for
the whole session. Registration now clears it, for inbound peers too: if they
reached us, our timeouts were about their NAT, not a dead host.

Retention was refreshed on every failure, so a row re-dialled at least once
per 30 min never aged out and its counter only climbed — rows reached
fail_count 107-109, a permanent sticky block on an endpoint that merely fails
intermittently. Escalation now counts failures within a 10 min streak window.

Sticky blocks were absolute, which is right while anything else is dialable and
starves the torrent once nothing is: 1151 of 1162 active blocks refused
resurrection, so a 50-endpoint batch request returned 3-8. Under min_count
pressure they are now last-resort, ordered productive, then soft before sticky,
then v6 before v4, then fewest failures.

Co-authored-by: Cursor <cursoragent@cursor.com>

Peer.Endpoints monitors the per-peer supervisor, and that supervisor is
auto_shutdown: :any_significant — it exits with a bare :shutdown as soon as
either child stops, whatever the child's reason was. So every disconnect logged
reason=:shutdown: true of the supervisor, and useless for the one question worth
asking when a swarm keeps shrinking, which is whether the peer left or we
dropped it.

The controller now hands its reason over from terminate/2, which runs before the
supervisor exits, so the note is in place when the :DOWN arrives. A note is
discarded when the endpoint registers again, since it describes the connection
that ended rather than the one starting.

Co-authored-by: Cursor <cursoragent@cursor.com>

The blacklist was a MapSet that only grew, so one bad frame excluded a peer for
the rest of the session, across every torrent. A ten-minute live run banned 352
peer IDs while eight of nine torrents could not get past three connections — on
a CGNAT host, where a torrent runs on 1-8 peers, that list *is* the swarm.

A ban is a hedge against a peer that will misbehave again, and re-testing it
costs one connection, so it does not need to outlive the failure by much: 30 min
with a cap, swept in the background. Each ban now also logs the rule that fired,
because "protocol_error" alone gave no way to tell a broken client from a rule of
ours that is too strict.

Co-authored-by: Cursor <cursoragent@cursor.com>

BEP 6 sends have_all, have_none, suggest_piece, allowed_fast and reject only once
both sides advertised the Fast extension, and we do advertise it, so a peer
sending one without it is at odds with the spec. Dropping the connection over it
was still wrong, because terminate/2 also blacklists the peer ID for every
torrent: 71 of 98 disconnects in one three-minute window, all of them qBittorrent
and Transmission builds that do support Fast.

have_all and have_none now go through the normal handler — they carry what a full
or empty bitfield carries, and a seeder we record as having nothing is a
connection we can never request from. The rest are advisory, or self-healing in
reject's case since the request timeout already covers it, so they are logged and
ignored.

A frame that stops half-way is a congested path, not a slow-loris: a uTP window
that collapsed, a peer that swapped. Dropping the connection is the defence
there; the ban only removed a mainstream client from every torrent at once.

Co-authored-by: Cursor <cursoragent@cursor.com>

A protocol error drops the peer and blacklists its ID for every torrent, but the
log only carried the verdict, and a dozen rules share it. A run that banned 100
peers in five minutes gave no way to tell which one fired — so no way to tell a
broken client from a rule of ours that is too strict.

The generic cast handler now logs the wire message that was rejected, and the
piece-bounds check logs the block it refused alongside the torrent's own geometry,
which is what makes an off-by-one in our bookkeeping visible.

Also updates the DialBackoff state-machine model to the current resurrection
order: sticky blocks are last-resort now, not excluded.

Co-authored-by: Cursor <cursoragent@cursor.com>

A cancel is not atomic. When we cancel a block, get choked, or repin a peer to
another piece, we dropped the request locally the same instant — but the block
may already be on the wire, and BEP 6 requires the peer to answer every request
exactly once, so a cancelled request comes back as a piece or a reject one RTT
later. Both landed in the "not outstanding" branch, which returned
:protocol_error: disconnect plus a blacklist of the peer ID across every
torrent.

So the scheduler was feeding the blacklist. Every repin opened a window in which
the peer's correct answer got it banned, and the Fast clients — the ones obliged
to send the reject — were hit hardest. A five-minute window showed 46 bans from
handle_piece and 29 from handle_reject, with the swarm never growing past a
handful of peers.

Withdrawn blocks are now remembered for a bounded window: an answer for one is
expected, and a piece is still counted since the payload reaches storage either
way. A block matching neither set is unrequested — wasteful rather than
malicious, so it ends the connection at 512 blocks without banning the ID.

Co-authored-by: Cursor <cursoragent@cursor.com>

The window was cleared wholesale once it passed its cap, because a MapSet has no
order to evict by. A peer that gets repinned often withdraws far more blocks
than the window holds, so the clear kept landing on blocks that were still
genuinely in flight: a four-minute run logged 4392 answers as unsolicited, and
six peers crossed the disconnect threshold on nothing but their own correct
rejects.

Retiring a full generation instead keeps at least one cap's worth of recent
withdrawals recognisable at all times, and eviction now drops the oldest rather
than everything.

Co-authored-by: Cursor <cursoragent@cursor.com>

The withdrawn-request window was a set, so it could only record *that* a block
had been withdrawn, not how many answers were still owed for it. Live logs show
the same block requested, cancelled and requested again up to seven times
against one peer, and the first answer erased the block from the window, so
every later one read as unsolicited: 71% of the 3891 "unsolicited" answers in a
25-minute run were for blocks we had demonstrably requested from that same peer.

BEP 6 gives an exact rule to count with — one answer per request, always — so
the window is now a count per block, decremented as answers arrive. That also
makes the bookkeeping independent of arrival order, which matters because a
re-requested block sits in both the live set and the window at once.

Co-authored-by: Cursor <cursoragent@cursor.com>

…t peer

A dial batch resolved only when its slowest endpoint did, and one endpoint can
hold a slot for the whole connect + handshake budget: measured successes came
back at 16s, 42s and 42s, with a worst case near 55s. Until then the manager
refused to start anything new for that torrent.

That is worst exactly where it hurts. A starved torrent whose queue is all IPv4
gets capped to a four-endpoint probe batch, so it was making four dial attempts
per minute against a CGNAT success rate near 1% — one torrent sat at zero peers
for the whole run with 18 candidates queued. The torrents that did fine were the
ones with deep enough queues to fill a 40-endpoint batch.

Batches may now overlap, bounded by endpoints in flight (40) and concurrent
batches (3), with in-flight endpoints excluded from selection so overlapping
batches cannot dial the same peer twice. `dialing?` keeps its meaning as a hard
stop and is now set only at saturation.

Co-authored-by: Cursor <cursoragent@cursor.com>

A pin counted as "drained" whenever the piece had no unclaimed blocks
left, without asking who had claimed them. A peer that had claimed every
remaining block on its piece therefore made that piece look drained,
was moved off it, and the move cancelled the very requests that drained
it: the blocks returned to `waiting`, the piece looked workable again,
and the next reconcile tick pinned the peer straight back.

That oscillated at the 2 s tick. One peer re-sent the same block 79
times in ten minutes and delivered none of it, and across the swarm we
sent 17 wire requests per block received while the cancels churned the
withdrawn-answer window behind them.

A piece now counts as drained for a peer only when it has neither
unclaimed blocks nor in-flight requests from that peer, so the rule
still frees peers parked on someone else's in-flight blocks — its
original purpose — without evicting the peer doing the work.

Measured over seven minutes on the live node: requests per block
17:1 -> 1.1:1, re-request factor 5.66x -> 1.08x, worst peer 55.8x ->
1.0x, swarm 28 -> 73 peers, aggregate 1.79 -> 2.47 MB/s.

Co-authored-by: Cursor <cursoragent@cursor.com>

Two defects held peers on a piece that had nothing left for them, both
in the endgame path, and together they parked torrents just short of
completion.

First, `pin_drained?/4` branched on the *torrent's* mode. A torrent
enters `:endgame` while the piece workers it already started keep
running in normal mode, and for those the endgame branch probed
`piece_has_waiting?/2`, which counts blocks in flight to *other* peers.
A normal-mode worker never hands out a block that is already in flight,
so that is exactly the wrong answer: the pin was held for work this peer
could never be given. The probe is now `piece_serves_peer?/3` in both
modes, which is well-defined because `waiting` already encodes the mode
— a normal worker drops a block when it hands it out, an endgame worker
keeps it until delivery, so redundant re-requests stay possible and the
endgame pin stays useful until the piece is genuinely finished.

Second, `endgame_preferred_index/2` destructured the peer key as
`{_hash, peer_id}`, but `Peer.make_key/2` builds `{id, hash}`. It was
hashing the torrent hash, so every peer on a torrent got the same
answer: the mechanism meant to spread peers across all remaining indices
funnelled the whole swarm onto one. That gate now also guards the
drained-pin path, which endgame needs because `target_accepts_repin?/3`
accepts any active index there — without it the peers freed by the first
fix would all follow `reconcile_refresh_interest/4` onto whichever index
it walks first.

Measured live on a torrent stopped at 99.18% for over fifteen minutes:
8 of its 10 peers pinned to one piece with no unclaimed blocks left,
three of them unchoked and idle, while six pieces sat with all 64 blocks
unclaimed and no peer at all.

Co-authored-by: Cursor <cursoragent@cursor.com>

`stale_useless_pin?/1` required `choke_me`, so the only pin it ever
released belonged to a peer the remote had choked — and a choked peer
holds no requests at all, because `handle_choke/1` clears them. The
harmful case was the one it skipped: an unchoked peer keeps a full
64-request pipeline, so every block it sits on is unavailable to anyone
else and is merely re-timed-out every `@timeout_request` and handed
straight back to it.

Live on a torrent stopped at 99%: two peers held one whole piece each,
61 and 64 blocks in flight, having delivered 48 KiB and 0 B, while two
pieces with all 64 blocks unclaimed had no peer at all. The same two
peers re-requested their 64 blocks 821 and 622 times in five minutes,
the interval alternating ~30 s / ~24 s — the block timeout reclaiming
them and the pump returning them to the peer that had just failed them.

Zero bytes on the pin now releases it whichever way the choke went, on a
longer threshold when unchoked (60 s) than when choked (15-20 s): longer
than a block timeout, so a merely slow peer gets a full request cycle to
produce something before it loses the pin. This is what other clients
call snubbing.

Co-authored-by: Cursor <cursoragent@cursor.com>

…sition

A piece worker reads the torrent's mode once, in State.download/3, so a piece
that was already downloading when the torrent crossed the endgame threshold ran
in normal mode for the rest of its life -- and normal mode gives a block to
exactly one peer. Redundancy was therefore absent from precisely the pieces
endgame exists for.

Live: a torrent sat at 99.939% for over an hour on one 1 MiB piece. The worker
was mode=nil under an :endgame torrent with waiting=[] and all 27 remaining
blocks in flight to a single peer that had logged 297 request timeouts on that
piece and zero hash failures, while three unchoked peers holding the piece had
nothing they were allowed to ask for. Each block was only retried on its own
30s timeout and handed straight back to the peer that had just failed it.

:reconcile_pump now upgrades active workers level-triggered. enter_endgame/1 is
idempotent and one-way, and re-queues the in-flight subpieces into waiting so
other peers may request them while the requests already out stay in place --
endgame adds sources for a block rather than taking it from the peer fetching
it. The pending stall/orphan timer is cancelled and flushed, since delivered
late it would abort a worker that has just started making progress. Redundancy
stays capped at @endgame_redundancy, and pieces started after the transition
already behaved this way.

Co-authored-by: Cursor <cursoragent@cursor.com>

Swarm health under CGNAT. Three separate mechanisms were shedding peers a
handful-of-peers swarm cannot afford — cancelled requests answered correctly and
punished as protocol errors, Fast messages treated the same way, and a blacklist
that never expired — while the piece scheduler parked the survivors on work they
were not allowed to do, leaving torrents at 99% with unchoked idle peers beside
unclaimed blocks. Also: endgame now reaches pieces already in flight, dial
batches overlap instead of serialising on their slowest endpoint, and writes to
a peer that stopped reading are bounded.

The peer ID prefix is derived from the package version, so this also moves the
advertised BEP 20 prefix from ET0-6-5 to ET0-6-6 — recorded in README and
PROTOCOL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Overlapping dial batches pushed two functions past Credo's ABC ceiling of 30,
which fails the strict Credo gate: dial_batch/2 reached 42, because selecting
the peers, the empty-queue replenish branch and the task launch all shared one
body, and the {:dial_done, ...} clause reached 35 for rebuilding six state
fields inline.

No behaviour change. select_dial_peers/3 and launch_dial_batch/3 split
dial_batch/2 along the branch it already had, and release_dial_batch/3 takes the
state rebuild out of handle_info/2 — which also gives the rule a name: a failed
endpoint stays queued so backoff ordering decides when to retry it, while the
rest leave the queue with the batch, and the slots are freed whatever the
results were so the next batch can start.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mix sobelow --out writes sobelow.sarif into the repo root. CI regenerates it per
run on a throwaway runner, but reproducing the CI gates locally before a PR
leaves it in the working tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>